Papers with cross-lingual representation
Do Explicit Alignments Robustly Improve Multilingual Encoders? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Explicit alignment objectives based on bitexts like Europarl and MultiUN have been shown to improve cross-lingual representations. |
| Approach: | They propose a new contrastive alignment objective that can better utilize bitexts . they propose to use a random sample of 1 million pair subset of OPUS data . |
| Outcome: | The proposed objective outperforms existing alignment objectives on a random 1 million pair subset of the OPUS dataset. |
Toward the Limitation of Code-Switching in Cross-Lingual Transfer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown the success of multilingual pretrained models for cross-lingual knowledge transfer. |
| Approach: | They propose to make code-switched sentences replace tokens from multiple languages so they are grammatically consistent . they also consider the similarity between context and the switched tokens to ensure that the newly substituted sentences are grammatically consistent - a limitation that could affect inference . |
| Outcome: | The proposed method outperforms the mBERT and original code-switching method on cross-lingual POS and Named-Entity-Recognition tasks on 30+ languages. |